Exploring Semantic Concept Using Local Invariant Features

This paper studies the role and performance of local invariant features arisen from interest points in describing and sketching semantic concepts. Both the local description and spatial location of interest points are exploited, separately and jointly, for concept-based retrieval. In concept description, a visual dictionary is generated with each keyframe being depicted as a vector of keywords. Semantic concepts are learnt and then spotted in this vector space model. In concept sketching, the location distribution of interest points, which outlines the basic shape of concepts, is novelly modelled with embedded Earth Mover's Distance. Experimental results with TRECVID-2005 corpus show that by incorporating both properties of interest points with baseline features, an improvement of 70% (over color) and 26% (over color and texture) in concept retrieval is reported.